1. Field
The present invention relates generally to database backup systems, and more specifically to an apparatus and method of replicating and storing backup databases in real-time or near real-time.
2. Background
PourOver is a method and system that replicates continually generated transaction data to a backup location in order to achieve real-time or near real-time database consistency between the active and backup databases. It can be used to construct a consistent backup database that is up-to-date (real-time) or within a few minutes' worth of transactions (near real-time). The backup location can be situated at a remote site.
High-availability systems that handle millions of transactions per day require that the data be reliably and continuously replicated to backup locations. This is so in the situation of a disaster where the running production site has failed or even potentially become inaccessible, the backup site can be brought up with no loss (real-time) or almost no loss of transaction data (near real-time).
An ideal system is a real-time system that operates constantly with extremely low downtime requirements, where inter-related data is stored across multiple files and distributed across multiple machines and disk drives. Further, due to high availability requirements a remote backup site needs to be used that replicates the primary site. When failing over from one site to another, either as a planned and/or unplanned action, the files need to be replicated in a consistent manner on the backup site in real or near real time with minimal or no data loss. Once, the files are restored on the backup site, that site can preferably take over operation from the site that was previously being backed up.
Presently, there are some commercial databases that implement database replication across distributed locations. However, they do not handle the general case of replicating arbitrary inter-related files, across multiple file-systems/machines/disks. In addition, some operating systems (and disk subsystems) can mirror data across distributed sites. This requires that all the files be on the same mirror, that the mirror gets all the writes in order from the operating system and that the mirror replicates in the order of the writes to ensure that all the files are consistent. Typically, when a transaction is lost the mirror needs to be rebuilt and while in the rebuilding process the mirrored site may be in an inconsistent state and may not be available. Some journaled file systems implement the required functionality, but not on a distributed basis.
Some of these prior art systems include fault tolerant clustered file systems, such as Calypso as described at http://www.research.ibm.com/caly, Sprite as described at http://portal.acm.org/citation.cfm?id=42183 and AFS as described at http://portal.acm.org/citation.cfm?doid=35037.35059. These systems implement the required functionality within a cluster but not across clusters. Veritas Cluster Server, as described at http://www.symantec.com/enterprise/products/overview.jsp?pcid=1019&pvid=20—1, implements the required functionality by combining a Veritas file system and volume replicator. The Veritas Cluster Server uses a Veritas file system to track all file modifications and uses a volume replicator to replicate any modified data to the remote cluster. The replicated data is not journaled, but is applied immediately to the remote cluster. Pratima, as described at http://www.linuxjournal.com/article/7265, and DRDB, as described at http://www.linbit.com/en/drbd, implements the required functionality at the block level using a customized Linux device driver. The Pratima and DRDB systems use custom Linux device drivers to replicate block updates across remote clusters. The replicated data is not journaled, but is applied immediately to the remote cluster.
There are several disadvantages and shortcomings of these prior art systems. The Veritas Cluster Server, Pratima and DRDB systems are volume based (not file based). In addition, these systems do not maintain journals, so during a rebuild, the remote cluster is not in a consistent state and cannot be used as a backup. This creates a potentially large window in time in which no backup cluster is available for fail over and potentially cannot be made consistent in the event of a network failure during this window (this can be overcome by replicating to a third location, but requires a third copy of the data and the network resources to make the extra copy). The Veritas Cluster Server requires a Veritas file system and is not currently available for virtual memory system (VMS) and the Pratima and DRDB systems require a Linux O/S and a customized device driver and is also is not available for VMS. These systems require the use of proprietary databases and specialized hardware. Further, these systems are not universal. They cannot back up multiple inter-related files across file systems or machines, they cannot solve the problem of retaining the ability to use the previous backup while rebuilding (unless two backup sites are used and only one site is rebuilt at a time) and they do not ensure file consistency to a specific point in time